Active Learning with Oracle Epiphany
نویسندگان
چکیده
We present a theoretical analysis of active learning with more realistic interactions with human oracles. Previous empirical studies have shown oracles abstaining on difficult queries until accumulating enough information to make label decisions. We formalize this phenomenon with an “oracle epiphany model” and analyze active learning query complexity under such oracles for both the realizable and the agnostic cases. Our analysis shows that active learning is possible with oracle epiphany, but incurs an additional cost depending on when the epiphany happens. Our results suggest new, principled active learning approaches with realistic oracles.
منابع مشابه
Adaptive Proactive Learning with Cost-Reliability Tradeoff
Proactive Learning is a generalized form of active learning where the learner must reach out to multiple oracles exhibiting different costs and reliabilities (label noise). One of the its major goals is to capture the cost-noise tradeoff in oracle selection. Sequential active learning exhibits coarse accuracy at the beginning and progressively refine prediction at later stages. The ability to l...
متن کاملActive Learning for Hierarchical Text Classification
Hierarchical text classification plays an important role in many real-world applications, such as webpage topic classification, product categorization and user feedback classification. Usually a large number of training examples are needed to build an accurate hierarchical classification system. Active learning has been shown to reduce the training examples significantly, but it has not been ap...
متن کاملActive Learning with Expert Advice
Conventional learning with expert advice methods assume a learner is always receiving the outcome (e.g., class labels) of every incoming training instance at the end of each trial. In real applications, acquiring the outcome from oracle can be costly or time consuming. In this paper, we address a new problem of active learning with expert advice, where the outcome of an instance is disclosed on...
متن کاملActive Learning from Crowds
Obtaining labels can be expensive or timeconsuming, but unlabeled data is often abundant and easier to obtain. Most learning tasks can be made more efficient, in terms of labeling cost, by intelligently choosing specific unlabeled instances to be labeled by an oracle. The general problem of optimally choosing these instances is known as active learning. As it is usually set in the context of su...
متن کاملActive Learning from Weak and Strong Labelers
An active learner is given a hypothesis class, a large set of unlabeled examples and the ability to interactively query labels to an oracle of a subset of these examples; the goal of the learner is to learn a hypothesis in the class that fits the data well by making as few label queries as possible. This work addresses active learning with labels obtained from strong and weak labelers, where in...
متن کامل